Skip to content

Use Ec2MetadataClient in defaults mode #6301

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: feature/master/use-imds-client
Choose a base branch
from

Conversation

S-Saranya1
Copy link
Contributor

@S-Saranya1 S-Saranya1 commented Jul 28, 2025

AutoDefaultsModeDiscovery changes to use Ec2MetadataClient

Motivation and Context

This change migrates IMDS-backed providers from using internal utilities to the public Ec2MetadataClient API.

  • AutoDefaultsModeDiscovery currently uses the internal EC2MetadataUtils class via HttpResourcesUtils, which doesn't respect certain IMDS client configurations
  • Lacks features like IMDS session token caching and auto-refresh
  • It replaces the basic HttpURLConnection with a HTTP client that offers much better debugging capabilities
  • Uses separate IMDS implementation instead of the public Ec2MetadataClient

This PR specifically addresses the Auto defaults mode discovery component, which is one of three planned migrations (along with InstanceProfileCredentialsProvider and InstanceProfileRegionProvider).

Github Feature Request: #5876

Modifications

  • Migrated AutoDefaultsModeDiscovery class to use Ec2MetadataClient from EC2MetadataUtils.
  • Created Ec2MetadataSharedClient, which creates a shared HTTP client that is shared among the providers.
  • Added DefaultEc2MetadataClientWithFallback for IMDSv1 compatibility.
  • Added imds module dependency to aws-core module's pom.xml

Testing

  • All existing tests pass - AutoDefaultsModeDiscoveryTest continues to validate original functionality
  • Added "AutoDefaultsModeDiscoveryEc2MetadataClientTest" which has unit tests that test the migration from EC2MetadataUtils to Ec2MetadataClient.

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@S-Saranya1 S-Saranya1 requested a review from a team as a code owner July 28, 2025 19:42
Adding the logic to close the client
Addressing PR feedback
return Optional.ofNullable(ec2InstanceRegion);
} catch (Exception exception) {
} catch (SdkClientException e) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was this change for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To fix a SpotBugs warning that complained about catching Exception when no exception was thrown and client.get() actually throws SdkClientException (and its subclasses), so changed this to more specific exception.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do RuntimeException, instead? This would catch the same exceptions as before, but not upset spotbugs. Unless we want a potential behavior change?

return this;
}

public synchronized Ec2MetadataClient build() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This synchronized uses a different lock from the containing class's synchronized. Maybe we should just manually create a reentrant lock and share that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, Thanks for this, updated to use reentrant lock.

* This provides resource efficiency by sharing a single HTTP client across all IMDS-backed providers
*/
@SdkProtectedApi
public final class Ec2MetadataSharedClient {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of having a separate protected class and API for this, should it be a feature of Ec2MetadataClient? E.g. if they do Ec2MetadataClient.builder().build(), we use a shared HTTP client instead of a new one? We'd still use reference counting to close the shared HTTP client when no metadata clients are using it anymore.

That would remove the need for another protected API we need to maintain/remember to use, and improve the creation time of multiple Ec2MetadataClients for customers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we make Ec2MetadataClient.builder().build() calls use a shared HTTP client, then client.close() would only decrement the reference count instead of actually closing resources, this would be a behavior change for users right? They expect their client to be fully closed but it's not closing. Alternatively, if we kept user-created clients separate from internal provider clients, we'd need to add a new public API feature to Ec2MetadataClient for providers to use shared clients, but that would be a public API change and users could also access.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we changed the default to use a shared client, how would that behavior change negatively affect the customer?

Comment on lines +54 to +56
* An Implementation of the Ec2Metadata Interface with IMDSv1 fallback support.
* This client is identical to DefaultEc2MetadataClient but provides automatic fallback
* to IMDSv1 when IMDSv2 token retrieval fails (except for 400 errors).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this just be a feature of Ec2MetadataClient as well? Ec2MetadataClient.builder().v1FallbackAllowed(true).build()

It would remove the duplicated code between the two implementations, make the usage of v1 something customers can choose to use in their own Ec2MetadataClients, and reduce the number of APIs we need to manage/remember to use.

Copy link
Contributor Author

@S-Saranya1 S-Saranya1 Aug 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be a public API change, and we only want IMDSv1 fallback for backward compatibility in internal providers so didn't go for that approach.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can reduce the number of protected APIs and simplify the code, it might be worth having v1 fallback be a public API.

I'm concerned about the duplication and additional protected surface area the current approach creates. Can you think of other ways to reduce it, without making a public API for enabling v1 fallback?

Comment on lines -27 to 28
@SdkInternalApi
@SdkProtectedApi
public final class DefaultSdkHttpClientBuilder implements SdkHttpClient.Builder {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Darn, I see we already released something that uses this, so this is just backfilling it to mark it as protected. If we needed it to be protected, we should have moved it out of the internal package or ideally not have made it protected. Too late for that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah :(
Added the comment about it.

Copy link

sonarqubecloud bot commented Aug 1, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
77.4% Coverage on New Code (required ≥ 80%)
8.5% Duplication on New Code (required ≤ 3%)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Refactoring BaseEc2MetadataClient constructor
Copy link
Contributor

@L-Applin L-Applin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matt still has some concerns, but I think they were all part of the design that we reviewed while he was on vacation. It's all compromises for not introducing the v1 fallback available as a public API on the IMDS, which is what we agreed on during design.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants